Skip to content

Official build: validate + promote with the newly built Arcade (3/3)#17080

Merged
mmitche merged 32 commits into
dotnet:mainfrom
mmitche:dev/arcade-validation-removal/official-build-validation
Jul 14, 2026
Merged

Official build: validate + promote with the newly built Arcade (3/3)#17080
mmitche merged 32 commits into
dotnet:mainfrom
mmitche:dev/arcade-validation-removal/official-build-validation

Conversation

@mmitche

@mmitche mmitche commented Jul 7, 2026

Copy link
Copy Markdown
Member

Part 3 of removing the arcade -> arcade-validation promotion/validation gate (#17046). Builds on #17066 and #17078. Main-only — release branches keep using arcade-validation.

What the official build validates about the newly produced SDK

build stage — builds/packs/publishes arcade with the current SDK; registers in BAR; post-build publishes assets to .NET Eng - Validation.

ValidateSdk stage (gated by runValidateSdk, default true) — three validations of the new SDK, in parallel:

  • Self-build (eng/validate-sdk.yml) — update-packagesource bumps to the new SDK, then cibuild rebuilds arcade with it (build + sign + pack + publish).
  • Signing (eng/validate-signing.yml) — stages a curated set of artifact types (test.deb/.rpm/.pkg/.vsix/.nupkg, nested containers, loose assemblies, ...) and signs them with the new SignTool (build -sign), on Windows (test + real), Linux, macOS. Ported from arcade-validation's Validate_Signing_*, with the new-SDK bump added.
  • Promotion (eng/validate-promotion.ymlvalidate-promotion.ps1) — promotes to General Testing from a branch bumped to the new arcade (uses the current build commit; darc picks the publishing infra version), so the promotion pipeline runs the new Tasks.Feed.

PromoteToLatest stage (main-only) — promotes to .NET Eng - Latest with the default arcade (promote-to-latest.ps1, --skip-assets-publishing). Runs on build success; gated on ValidateSdk only when it is enabled, so a build can still be promoted when validation is off.

Review feedback addressed

  • No RepoTests in the official build (they run in the PR build only).
  • Promotion test separated from the Latest promotion.
  • validate-promotion.ps1 uses the current build commit (no darc get-build) and doesn't pin --publishing-infra-version.
  • Removed usePrebuiltArcadeForPublishing.

⚠️ Review needed

The signing and promotion validations run only in the internal official build and cannot be validated locally or in PR. They need @dotnet/dnceng review + a real official-build run. Key assumptions: cross-platform update-packagesource for signing; Mac/Linux real-signing wiring; BAR id from ReleaseConfigs.txt; dnceng-build-rw-code-rw-wif for the mirror push; dependsOn: publish_using_darc; General Testing promotion not conflicting with the Validation publish (arcade-validation had pre-checks — may need porting).

Validated locally: all YAML parses; both promotion scripts parse.

Part of #17046.

mmitche and others added 2 commits July 7, 2026 14:28
…dateSdk switch

The ValidateSdk stage already self-bootstraps arcade with the newly produced SDK. Change its
rebuild from /p:Test=false to /p:RunNonHelixTestsOnly=true so it also runs the ported Arcade SDK
validation RepoTests against the new SDK, and publish their results (enablePublishTestResults).

Add a runValidateSdk escape hatch (default true), exposed at queue time in azure-pipelines.yml,
that gates the whole ValidateSdk stage and its place in post-build publishDependsOn so a
badly-broken arcade can still ship a fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…otnet#17046)

Add a main-only PromoteWithNewArcade stage to the official build that replaces the promotion
validation dotnet/arcade-validation performed. After the build's assets are published to
'.NET Eng - Validation', eng/validation/promote-with-new-arcade.ps1:
  1. creates a dev branch off the built commit and runs darc update-dependencies --id <BAR>
     to bump Arcade to the version this build produced, pushing it to the internal AzDO mirror;
  2. promotes the build to 'General Testing' with --source-branch <devBranch> so darc runs the
     promotion (publishing) pipeline from that branch -> PublishArtifactsInManifest / Tasks.Feed
     come from the NEWLY BUILT Arcade, validating promotion with the new Arcade;
  3. on success, adds the build to '.NET Eng - Latest' with --skip-assets-publishing;
  4. cleans up the dev branch.

The BAR build id is read from the ReleaseConfigs artifact; the AzDO mirror push uses a
WIF-derived code-rw token. A usePrebuiltArcadeForPublishing parameter (default true) reverts to
promoting with the current global.json Arcade.

Main only: release branches keep using arcade-validation for this gate.

NOTE: runs only in the internal official build; needs @dotnet/dnceng review and a real run to
validate (not testable locally or in PR).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 7, 2026 21:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR is part 3 of removing the arcade -> arcade-validation promotion/validation gate by moving (1) SDK validation RepoTests into the internal official build’s ValidateSdk stage, and (2) main-branch promotion validation into a new stage that promotes using the newly built Arcade.

Changes:

  • Add a runValidateSdk queue-time parameter to optionally gate the ValidateSdk stage and its inclusion in post-build publishing dependencies.
  • Enable publishing of inline (non-Helix) RepoTests results from the ValidateSdk stage.
  • Add a main-only PromoteWithNewArcade stage and a new eng/validation/promote-with-new-arcade.ps1 script to validate promotion using the newly built Arcade before adding the build to .NET Eng - Latest (with an escape hatch via usePrebuiltArcadeForPublishing).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
eng/validation/promote-with-new-arcade.ps1 New promotion-validation script that creates a temporary branch bumped to the newly built Arcade and promotes from that branch.
eng/validate-sdk.yml Enables publishing test results for the inline validation RepoTests executed in the ValidateSdk job.
eng/build.yml Adds parameters and wires ValidateSdk gating plus a main-only PromoteWithNewArcade stage into the official pipeline.
azure-pipelines.yml Exposes queue-time parameters to control ValidateSdk and “use prebuilt Arcade for publishing” behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread eng/validation/promote-with-new-arcade.ps1 Outdated
Comment thread eng/validation/promote-with-new-arcade.ps1 Outdated
Comment thread eng/validation/promote-with-new-arcade.ps1 Outdated
Comment thread eng/validation/promote-with-new-arcade.ps1 Outdated
ViktorHofer
ViktorHofer previously approved these changes Jul 8, 2026
…eparate Latest promotion

Addresses review feedback on the official-build validation design (dotnet#17046):

- Do NOT run the RepoTests in the official build (revert ValidateSdk to /p:Test=false); the
  RepoTests run in the PR build only.
- ValidateSdk stage now runs three validations of the newly produced SDK in parallel:
  * self-build  (eng/validate-sdk.yml)      - rebuild arcade with the new SDK,
  * signing     (eng/validate-signing.yml)  - sign a curated set of artifact types with the new
                                              SignTool (ported arcade-validation Resources + jobs,
                                              cross-OS, after update-packagesource bumps the new SDK),
  * promotion   (eng/validate-promotion.yml)- validate promotion works with the new arcade.
- Separate the promotion *test* from the final promotion:
  * eng/validation/validate-promotion.ps1 promotes to 'General Testing' from a branch bumped to the
    new arcade (uses the current build commit; no --publishing-infra-version) - part of ValidateSdk.
  * eng/validation/promote-to-latest.ps1 promotes to '.NET Eng - Latest' with the default arcade in a
    separate PromoteToLatest stage that runs on build success, gated on ValidateSdk only when it is
    enabled - so a build can still be promoted when validation is disabled.
- Remove the usePrebuiltArcadeForPublishing parameter (the new-arcade-ness now lives only in the
  promotion test).

Main only: release branches keep using dotnet/arcade-validation for promotion.

NOTE: the signing and promotion validations run only in the internal official build and cannot be
validated locally or in PR; they need @dotnet/dnceng review and a real run.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 9, 2026 20:28
@mmitche

mmitche commented Jul 9, 2026

Copy link
Copy Markdown
Member Author

Reworked per review:

  1. No RepoTests in the official build — reverted the ValidateSdk rebuild to /p:Test=false.
  2. Signing validation with the new SDK added (eng/validate-signing.yml + ported resources) — signs a curated set of artifact types with the newly built SignTool across Windows/Linux/macOS.
  3. ValidateSdk = 3 parallel jobs: self-build, signing, promotion validation.
  4. Promotion test separated from the Latest promotion: validate-promotion.ps1 (→ General Testing, new arcade) runs inside ValidateSdk; promote-to-latest.ps1 (→ .NET Eng - Latest, default arcade) is a separate stage that runs on build success, gated on ValidateSdk only when enabled.
  5. Script fixes: uses the current build commit (no darc get-build); no --publishing-infra-version; removed usePrebuiltArcadeForPublishing.

Matches the agreed flow: build → publish to BAR → (if enabled) ValidateSdk {signing, self-build, promotion to General Testing with new arcade} → promote to .NET Eng - Latest with default arcade on success.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 23 changed files in this pull request and generated 6 comments.

Comment thread eng/build.yml
Comment thread eng/validation/validate-promotion.ps1 Outdated
Comment thread eng/validate-signing.yml
Comment thread eng/validate-signing.yml
Comment thread eng/validate-signing.yml
Comment thread eng/build.yml
ViktorHofer
ViktorHofer previously approved these changes Jul 10, 2026
Copilot AI review requested due to automatic review settings July 10, 2026 15:43
The signing-validation inputs already exist as SignTool unit-test resources
(src/Microsoft.DotNet.SignTool.Tests/Resources), which is a superset of arcade-validation's set.
Point eng/validate-signing.yml at that folder with a curated file list instead of duplicating the
binaries under eng/validation/resources, and remove the duplicated resources.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mmitche
mmitche force-pushed the dev/arcade-validation-removal/official-build-validation branch from 80e1f36 to e942b69 Compare July 10, 2026 15:43
@mmitche

mmitche commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

Per review: the signing-validation inputs already exist as SignTool unit-test resources (src/Microsoft.DotNet.SignTool.Tests/Resources) — a superset of arcade-validation's set. So instead of duplicating binaries under eng/, I removed eng/validation/resources and pointed eng/validate-signing.yml at the existing SignTool.Tests/Resources with a curated file list (ContainerOne/NestedContainer/PackageWithZip nupkgs, test.deb/.rpm/.pkg/.mpack/.zip/.tgz, SignedLibrary.dll/EmptyPKT.dll/CustomTargetFrameworkAttribute.dll, and the .vsixes). No new binary assets are added.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Comment thread eng/validate-signing.yml Outdated
Comment thread eng/validate-signing.yml Outdated
Comment thread eng/validate-signing.yml Outdated
Comment thread eng/validate-signing.yml Outdated
Copilot AI review requested due to automatic review settings July 10, 2026 15:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Comment thread eng/validation/validate-promotion.ps1 Outdated
The VSIX staging step was only on the Windows signing job; arcade-validation copies the .vsix
inputs in all three OS jobs. Add it to Linux and macOS so their sign runs also exercise VSIX signing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 10, 2026 16:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Comment thread eng/validate-signing.yml Outdated
Comment thread eng/validate-signing.yml Outdated
Comment thread eng/validation/validate-promotion.ps1 Outdated
Copilot AI review requested due to automatic review settings July 10, 2026 16:13
…a parameters

- promote-to-latest.ps1 was essentially a single 'darc add-build-to-channel' call; inline it into the
  PromoteToLatest stage (Get-Darc from tools.ps1) and remove the script.
- Hoist the signing-validation package/VSIX copy lists into template parameters
  (signingTestPackages / signingTestVSIXes) so they are defined once and shared by all three jobs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@mmitche
mmitche force-pushed the dev/arcade-validation-removal/official-build-validation branch from d68bb46 to efeb511 Compare July 10, 2026 16:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Comment thread eng/validation/validate-promotion.ps1
Comment thread eng/validation/validate-promotion.ps1
Comment thread eng/validation/use-built-sdk.ps1
ViktorHofer
ViktorHofer previously approved these changes Jul 13, 2026
darc update-dependencies --packages-folder scans NON-recursively
(Directory.GetFiles(path, "*.nupkg")), but arcade's build artifacts nest the
produced packages under packages/<config>/NonShipping. Pointing darc at the
build_stage_artifacts root therefore found zero nupkgs and reported "Found no
dependencies to update" (leaving global.json on the bootstrap SDK).

Recursively locate the produced Microsoft.DotNet.Arcade.Sdk / Helix.Sdk nupkgs
(arcade only produces nonshipping packages; these are the only arcade-produced
deps tracked in Version.Details.xml), copy them into a single flat temp folder,
and point darc at that. Clean up the temp folder afterward.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 13, 2026 19:52
Simpler than gathering into a temp flat folder: arcade produces only
nonshipping packages, so Arcade.Sdk and Helix.Sdk are both in the same
packages/<config>/NonShipping folder. Point darc's (non-recursive)
--packages-folder at the directory containing the produced Arcade.Sdk nupkg.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Comment thread eng/validation/use-built-sdk.ps1
Copilot AI review requested due to automatic review settings July 13, 2026 19:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Comment thread eng/validation/use-built-sdk.ps1
darc update-dependencies resolves asset locations from Maestro/BAR, so it needs
an Azure AD token. In a plain powershell step darc's AzureCliCredential timed
out ("Azure CLI authentication timed out") because there was no az login
context. Run the darc step under AzureCLI@2 with the "Darc: Maestro Production"
service connection (same pattern as the promotion validation) so darc can
authenticate.

Note: the nonshipping-folder fix worked - darc found the packages and got past
"no dependencies to update"; this addresses the next blocker (auth).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 13, 2026 20:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Comment thread eng/validation/validate-promotion.ps1
Comment thread eng/validation/use-built-sdk.ps1
Testing confirmed darc update-dependencies --packages-folder DOES bump
global.json (Arcade.Sdk/Helix.Sdk are tracked in Version.Details.xml), but it
also re-syncs eng/common from the remote arcade repo at the built commit, which
requires a GitHub PAT and the commit to exist on github.com/dotnet/arcade. That
makes it work only on official main builds, not dev/PR builds.

Per that finding, keep the self-build on the file-based use-built-sdk.ps1 bump
(regex bump of global.json + local feed), which works identically on main and
PR with no darc/BAR/GitHub/network dependency. Remove the -AddFeedOnly switch
(now unused) and the darc-based update-sdk-versions.ps1 helper.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 13, 2026 22:50

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Comment thread eng/validation/use-built-sdk.ps1
use-built-sdk.ps1:
- Guard against a missing <packageSources> element with a clear error.
- Make the local-feed injection idempotent: remove any existing arcade-local-*
  <add>/<packageSource> entries before adding, so re-running in the same
  workspace doesn't create duplicate keys (which NuGet rejects).

validate-promotion.ps1:
- Throw a clear error if `darc get-build` returns an empty result (avoids a
  StrictMode null-reference later in the channel gate).
- Force-push the throwaway per-build validation branch so a leftover branch
  from a prior failed run doesn't cause a non-fast-forward push failure.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 13, 2026 22:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Comment thread eng/validate-signing.yml
Comment thread eng/build.yml Outdated
The target channel ('.NET Eng - Latest') was duplicated across the promotion
stage/job/task displayNames and the darc add-build-to-channel command, and the
promotion source branch ('refs/heads/main') was inlined in the stage gate.
Move both to top-level parameters (promotionChannel, promotionSourceBranch) so
each value is defined once and reused, per PR review feedback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 14, 2026 14:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Comment thread eng/build.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants